Comments for MEDB 5501, Week 7

Two-sample t-test (Independent-samples t-test)

Randomized trial
- Convenience sample
- Random assignment to treatment, control
- Measure continuous outcome
Cohort design
- Observe exposed and control subjects
- No random assignment
- Measure continuous outcome

Assumptions, 1 of 3

Group 1, 2 both normally distributed
- Assessed with histograms, boxplots, Q-Q plots
  - Assess each group separately, or
  - Combine after subtracting means
- Or rely on Central Limit Theorem

Assumptions, 2 of 3

Possibly different means, but same variance
- Compare the box part of the box plots
  - Interquartile range
  - Look for large disparities only (2 or 3 fold)
- Calculate and compare the standard deviations
  - Again, large disparities only
- Levene’s test (not recommended)
  - Too little power for small sample sizes and too much power for large sample sizes
  - Very sensitive to normality assumption
The equal variance assumption is important mostly for unequal sample sizes.

Assumptions, 3 of 3

Observations are independent
- Between groups
  - No matching
  - No longitudinal measures
- Within groups
  - No cluster effects
  - No infectious spread
- Assessed qualitatively

Independence is almost always assessed qualitatively. You have to assume indpendence between the two groups first. Data that is matched or paired violates this assumption (see the paired t-test later). You also can’t use the same subjects again. If you want to compare the same subjects at different time points that would be a violation of independeance.

You also have to assume independence within groups. I was doing a study of breast feeding where the researchers recruited 84 infants. There were 72 singleton births and 6 sets of twins. I bet the researchers were excited to get these twins: two observations for the price of a single consent form. But the measurement of breast feeding duration is not going to be independent. If one baby in the twin pair stops breast feeding, it increases the chances that the other baby will stop at the same time. It doesn’t happen all the time, but it happens enough that you can’t say that the data are independent.

Any time you have a clustering effect, measurements in the same family, for example, you should probably assume that the outcomes are not independent. If you conduct studies at multiple clinics, the observations of two patients at the same clinic are not going to be independent because the standard of care, skill of the medical personnel, and patient referral patterns are going to influence the results.

It’s okay if only a single clinic evaluates all of the patients. It’s only when there are multiple clinics that this may become an issue.

If you have clusters in your data, you can still analyze the data, but you have to account for the clusters in the model. Otherwise, the confidence intervals and hypothesis tests will be distorted.

Housing data dictionary, 1 of 5

source: 
  This file was found originally at a website 
  DASL (Data And Story Library) that is no 
  longer available. 

description:  
  The original source describes the data as
  "a random sample of records of resales of 
  homes from Feb 15 to Apr 30, 1993 from the
  files maintained by the Albuquerque Board 
  of Realtors. This type of data is 
  collected by multiple listing agencies in
  many cities and is used by realtors as an
  information base."

Housing data dictionary, 2 of 5

copyright:  
    Unknown. You should be able to use this data for
  individual educational purposes under the Fair Use
  guidelines of U.S. copyright law.

format: 
  delimiter: space
  varnames: first row of data
  missing-value-code: *
  rows: 117
  columns: 8

Housing data dictionary, 3 of 5

vars:
  Price:
    label: Selling price
    unit: dollars
    
  SquareFeet:
    label: Living space
    unit: square feet
    
  AgeYears:
    label: Age of home
    unit: years

Housing data dictionary, 4 of 5

  NumberFeatures:
    label: 
      Home features (dishwasher, refrigerator,
      microwave, disposer, washer, intercom, 
      skylight(s), compactor, dryer, handicap
      fit, cable TV access)
    scale: count
    range: 0 to 11  
    
  Northeast:
    label: Located in northeast sector of city?
    values:
      Yes: 1
      No: 0

Housing data dictionary, 5 of 5

  CustomBuild:
    label: Custom built?
    values:
      Yes: 1
      No: 0
    
  CornerLot:
    label: Corner location?
    values:
      Yes: 1
      No: 0

  Tax:
    label: Yearly property tax
    unit: dollars

Split file dialog box

Housing analysis

Price histogram, 1 of 2

Price histogram, 2 of 2

Stop grouping, using the Split file dialog box

Population pyramid

Agggregate data dialog box

Compute variable dialog box

Q-Q plot for all of the data

Boxplots

Should you consider a log transformation?

Yes because
- Data bounded below by zero
- Some evidence of non-normality
- Some evidence of unequal variation.
  - Unbalanced sample sizes
No because
- Some evidence of normality
  - Sample size is large
- Differences in variation not too extreme

Compute Variable dialog box

boxplots of the log transformed variables

Independent Samples T-Test dialog box

Define Groups dialog box

T-test output, 1 of 5

T-test output, 2 of 5

T-test output, 3 of 5

T-test output, 4 of 5

T-test output, 5 of 5

Univariate model dialog box

General linear model output, 1 of 3

General linear model output, 2 of 3

General linear model output, 3 of 3

What belongs in an interpretation

“We designed the study to have 90% power to detect a 4 degree difference between the groups in the increased range of elbow flexion. Alpha was set at 0.05. Patients receiving electrical stimulation (n=26) increased their range of elbow flexion by a mean of 16 degrees with a standard deviation of 4.5, whereas patients in the control group (n=25) increased their range of flexion by a mean of only 6.5 degrees with a standard deviation of 3.4. This 9.5 degree difference was statistically significant (95% CI = 7.23 to 11.73, two-tailed Student’s t test, t=8.43, P < 0.001).”

Lang and Secic, How to Report Statistics in Medicine, p47.

Power calculation, if possible

“We designed the study to have 90% power to detect a 4 degree difference between the groups in the increased range of elbow flexion. Alpha was set at 0.05. Patients receiving electrical stimulation (n=26) increased their range of elbow flexion by a mean of 16 degrees with a standard deviation of 4.5, whereas patients in the control group (n=25) increased their range of flexion by a mean of only 6.5 degrees with a standard deviation of 3.4. This 9.5 degree difference was statistically significant (95% CI = 7.23 to 11.73, two-tailed Student’s t test, t=8.43, P < 0.001).”

Lang and Secic, How to Report Statistics in Medicine, p47.

Alpha level

“We designed the study to have 90% power to detect a 4 degree difference between the groups in the increased range of elbow flexion. Alpha was set at 0.05. Patients receiving electrical stimulation (n=26) increased their range of elbow flexion by a mean of 16 degrees with a standard deviation of 4.5, whereas patients in the control group (n=25) increased their range of flexion by a mean of only 6.5 degrees with a standard deviation of 3.4. This 9.5 degree difference was statistically significant (95% CI = 7.23 to 11.73, two-tailed Student’s t test, t=8.43, P < 0.001).”

Lang and Secic, How to Report Statistics in Medicine, p47.

Sample statistics for each group

“We designed the study to have 90% power to detect a 4 degree difference between the groups in the increased range of elbow flexion. Alpha was set at 0.05. Patients receiving electrical stimulation (n=26) increased their range of elbow flexion by a mean of 16 degrees with a standard deviation of 4.5, whereas patients in the control group (n=25) increased their range of flexion by a mean of only 6.5 degrees with a standard deviation of 3.4. This 9.5 degree difference was statistically significant (95% CI = 7.23 to 11.73, two-tailed Student’s t test, t=8.43, P < 0.001).”

Lang and Secic, How to Report Statistics in Medicine, p47.

P-value AND confidence interval

“We designed the study to have 90% power to detect a 4 degree difference between the groups in the increased range of elbow flexion. Alpha was set at 0.05. Patients receiving electrical stimulation (n=26) increased their range of elbow flexion by a mean of 16 degrees with a standard deviation of 4.5, whereas patients in the control group (n=25) increased their range of flexion by a mean of only 6.5 degrees with a standard deviation of 3.4. This 9.5 degree difference was statistically significant (95% CI = 7.23 to 11.73, two-tailed Student’s t test, t=8.43, P < 0.001).”

Lang and Secic, How to Report Statistics in Medicine, p47.

One or two sided test

“We designed the study to have 90% power to detect a 4 degree difference between the groups in the increased range of elbow flexion. Alpha was set at 0.05. Patients receiving electrical stimulation (n=26) increased their range of elbow flexion by a mean of 16 degrees with a standard deviation of 4.5, whereas patients in the control group (n=25) increased their range of flexion by a mean of only 6.5 degrees with a standard deviation of 3.4. This 9.5 degree difference was statistically significant (95% CI = 7.23 to 11.73, two-tailed Student’s t test, t=8.43, P < 0.001).”

Lang and Secic, How to Report Statistics in Medicine, p47.

My interpretation

“Comparison of housing prices between groups used a two-sample t-test.”
“All tests were conducted using a two-sided alpha level of 0.05.”
“Houses that were custom built had an average price of 145 thousand dollars, which was 50 thousand dollars higher than the average price of normal houses. This difference was statistically significant (95% CI 36 thousand dollars to 64 thousand dollars, p=0.001).”

What I look for

The test you are using
Specify the number of tails
Round
Units of measure

Compare means dialog box

Compare means output

Geometric means

Geometric standard deviations

General linear model for log transformed data

Back-calculated statistics

Back calculated confidence intervals, 1 of 2

Back calculated confidence intervals, 2 of 2

Conceptual formula for sample size justification, 1 of 2

Conceptual formula for sample size justification, 2 of 2

Moon data dictionary, 1 of 4

---
data_dictionary: moon.txt

source:
  This data file is part of OzDASL, an archive of various data sets 
  useful for teaching. The maintainers of the data archive are from
  Australia, but this particular data set is not specific to that part
  of the world. The entire archive is at
  https://dasl.datadescription.com/

Moon data dictionary, 2 of 4

description:
  This data set shows a perceptual experiment where subjects were asked
  to estimate a size ratio with their head level to the ground and then
  with their head elevated (in other words, looking upward). Although
  the objects being compared were the same size, almost all subjects 
  overestimated the relative sizes. The hypothesis to be tested is 
  whether the overestimation is greater with eyes level than with eyes
  elevated. A more detailed description is available at
  https://gksmyth.github.io/ozdasl/general/moon.html
  
download:
  https://gksmyth.github.io/ozdasl/general/moon.txt

Moon data dictionary, 3 of 4

copyright:  
  Unknown. You should be able to use this data for individual 
  educational purposes under the Fair Use guidelines of U.S. 
  copyright law.

format: 
  delimiter: tab
  varnames: first row of data
  missing-value-code: not needed
  rows: 10
  columns: 3

Moon data dictionary, 4 of 4

vars:
  Subject:
    label: Subject number
    format: numeric

  Elevated:
    label: Perceived ratio with eyes elevated
    format: numeric
    
  Level:
    label: Perceived ratio with eyes level
    format: numeric
---

QQ plot of ratio under elevated eye condition

QQ plot of ratio under level eye condition

QQ plot of differences

Moon data, descriptive statistics

Paired Samples T Test dialog box

Paired T Test output, 1 of 5

Paired T Test output, 2 of 5

Paired T Test output, 3 of 5

Paired T Test output, 4 of 5

Paired T Test output, 5 of 5

My interpretation

“Comparison of viewing ratios between the eyes level and eyes elevated conditions used a paired t-test.”
“All tests were conducted using a two-sided alpha level of 0.05.”
“Viewing ratios under both conditions had comaparable sample means and standard deviations. The mean difference was 0.02 with a standard deviation of 0.14. This difference was not statistically significant (95% CI: -0.08 to 0.12, p=0.67).

Three things you need for a sample size justification

Hypothesis
Standard deviation
Minimum clinically important difference

If you work with a researcher who wants to justify a particular sample size, you need to get three things from them.

First, they need a research hypothesis. If they don’t have one, help them formulate one. Some research does not need a research hypothesis, of course. If you are conducting a descriptive study, try to find an important descriptive statistic and select a sample size so the confidence interval using that statistic is reasonably narrow. If your study has a qualitative goal, then you can use a qualitative justification for your sample size.

Second, they need to know how much noise there is in the data. This is measured by the standard deviation. They may have a standard deviation from a pilot study or from previous research. If they don’t have this, ask them for a paper that’s reasonably similar to theirs. It doesn’t have to be identical, but it should use the same outcome measure and the same types of patients. That paper should have a standard deviation in one of the tables.

If not, perhaps you can extrapolate from the range. The range divided by 4 (or maybe 6) is often a reasonable approximation to the standard deviation. You might be able to back-calculate the standard deviation from a confidence interval or p-value, but this is tricky. If all else fails, just use a reasonable guess.

Third, they need to know the minimum clinically important difference. This requires clinical judgement. How much of a change is large enough to get other clinicians to sit up and take notice?

Some people bypass the specification of the minimum clinically important difference and just say that they want to esimate a small, medium, or large effect size. This is a mistake. You cannot plan a study on the basis of a unitless measure.

Rule of 16

n per group = 16 / ES^2
Examples:
- ES = 0.5, n per group = 64
- ES = 0.1, n per group = 1,600

Comments for MEDB 5501, Week 7

Two-sample t-test (Independent-samples t-test)

Assumptions, 1 of 3

Assumptions, 2 of 3

Assumptions, 3 of 3

Housing data dictionary, 1 of 5

Housing data dictionary, 2 of 5

Housing data dictionary, 3 of 5

Housing data dictionary, 4 of 5

Housing data dictionary, 5 of 5

Split file dialog box

Housing analysis

Price histogram, 1 of 2

Price histogram, 2 of 2

Stop grouping, using the Split file dialog box

Population pyramid

Agggregate data dialog box

Compute variable dialog box

Q-Q plot for all of the data

Boxplots

Should you consider a log transformation?

Compute Variable dialog box

boxplots of the log transformed variables

Independent Samples T-Test dialog box

Define Groups dialog box

T-test output, 1 of 5

T-test output, 2 of 5

T-test output, 3 of 5

T-test output, 4 of 5

T-test output, 5 of 5

Univariate model dialog box

Univariate model dialog box

General linear model output, 1 of 3

General linear model output, 2 of 3

General linear model output, 3 of 3

What belongs in an interpretation

Power calculation, if possible

Alpha level

Sample statistics for each group

P-value AND confidence interval

One or two sided test

My interpretation

What I look for

Compare means dialog box

Compare means output

Geometric means

Geometric standard deviations

General linear model for log transformed data

Back-calculated statistics

Back calculated confidence intervals, 1 of 2

Back calculated confidence intervals, 2 of 2

Conceptual formula for sample size justification, 1 of 2

Conceptual formula for sample size justification, 2 of 2

Moon data dictionary, 1 of 4

Moon data dictionary, 2 of 4

Moon data dictionary, 3 of 4

Moon data dictionary, 4 of 4

QQ plot of ratio under elevated eye condition

QQ plot of ratio under level eye condition

QQ plot of differences

Moon data, descriptive statistics

Paired Samples T Test dialog box

Paired T Test output, 1 of 5

Paired T Test output, 2 of 5

Paired T Test output, 3 of 5

Paired T Test output, 4 of 5

Paired T Test output, 5 of 5

My interpretation

Three things you need for a sample size justification

Rule of 16

Sample size calculation dialog box

Sample size calculation output

Sample size calculation output for second scenario